Information Theory With Kernel Methods

We consider the analysis of probability distributions through their associated covariance operators from reproducing kernel Hilbert spaces. show that von Neumann entropy and relative these are intimately related to usual notions Shannon entropy, share many properties. They come together with efficient estimation algorithms various oracles on distributions. also product spaces for tensor kernels, we can define mutual information joint entropies, which then characterize independence perfectly, but only partially conditional independence. finally how new lead upper-bounds log partition functions, be used convex optimization within variational inference methods, providing a family probabilistic methods.